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ABSTRACT 



The present invention is an interactive computer system 
which may operate on a computer network. Subscribers 
interact with a fully interactive programthrough the use of 
input devices and a personal computer or a television. The 
multiple video/audio datastreams may be received from a 
broadcast transmission source or may be resident in local or 
external storage. In response to user inputs, a personalized 
graphics, video and/or audio presentation is provided to the 
user either immediately or at a later time. If not presented 
immediately, the interactive computer system utilizes "trig- 
ger points" to determine when to enable multiple multimedia 
segments during the show. The CPU uses embedded or 
stored authoring commands for integrating the various mul- 
timedia elements. The interactive multimedia computer 
enables seamless flicker-free switching from one signal to 
another on the same or different channels. 

19 Claims, 13 Drawing Sheets 
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INTERACTIVE COMPUTER SYSTEM FOR 
PROVIDING AN INTERACTIVE 
PRESENTATION WITH PERSONALIZED 

VIDEO, AUDIO AND GRAPHICS 
RESPONSES FOR MULTIPLE VIEWERS 

RELATED APPLICATIONS 

This is a continuation-in-part of application Ser. No. 
08/443,607, filed May 18, 1995 U.S. Pat. No. 5,724,091, 
which is continuation-in-part of application Ser. No. 08/166, 
608, filed Dec. 13, 1993, now abandoned which in turn is a 
continuation of application Ser. No. 07/797,298, filed Nov. 
25, 1991, now abandoned. The interactive television pro- 
gram is of a type disclosed in U.S. Pat. Nos. 5,585,858, 
4,847,700, 4,507,680, 4,573,072, 3,947,792, 4,602,279, 
4,264,925, and 4,264,924. 

BACKGROUND OF THE INVENTION 

Interactive video and audio presentation systems are 
currently being introduced into the entertainment and edu- 
cational industries. A prominent interactive technology that 
has been applied successfully in these industries is based on 
providing interactivity in a one-way system through the 
provision of multiple parallel channels of information. For 
example, commonly owned Freeman et al. patents, U.S. Pat. 
Nos. 4,264,925 and 4,264,924, which provide both audio 
and video interactivity, disclose interactive television sys- 
tems where switching among multiple broadcast or cable 
channels based on viewer selections provides an interactive 
capability. 

These systems have been enhanced to include memory 
functions using computer logic and memory, where selec- 
tion of system responses played to the viewer are based on 
the processing and storage of subscriber responses, as dis- 
closed in Freeman patent, U.S. Pat. No. 4,507,680. 

The benefits of providing interactivity through the use of 
different audio responses is disclosed in Freeman, U.S. Pat. 
Nos. 4,847,698, 4,847,699 and 4,847,700. These television 
systems provide a common video signal accompanied by 
several synchronized audio channels to provide content 
related user selectable responses. The audio signals produce 
different audio responses, and in some cases, these are 
syllable synched to a first audio script and to the video signal 
(such as to a person or character on a display), providing the 
perception that the person's or character's mouth move- 
ments match the spoken words. 

Interactivity is brought to the classroom in the Freeman 
U.S. patent application Ser. No. 08/228,355. The distance 
learning system claimed in this application enhances the 
classroom educational experience through an innovative use 
of interactive technology over transmission independent 
media. When an instructor, either broadcast live on video or 
displayed from videotape, asks a question, each and every 
student responds, preferably by entering a response on a 
remote handset, and each student immediately receives a 
distinct and substantive audio response to his or her unique 
selection. The individualization of audio response from the 
interactive program is a major aspect of the invention. 

Individualization of audio is brought to the home based on 
the technology disclosed in Freeman U.S. patent application 
Ser. No. 08/228,355. This system provides a program that 
can be watched on any conventional television set or mul- 
timedia computer as a normal program. But if the viewer has 
a special interactive program box connected to the 
television, he or she can experience a fully functional 
interactive program. Each interactive viewer enjoys person- 



51,881 

2 

alized audio responses and video graphics overlayed on the 
screen. The interactive program can be provided to televi- 
sion sets or to computers by cable, direct broadcast satellite, 
television broadcast or other transmission means, and can be 

5 analog or digital. Unlike previous interactive systems, this 
application covers a system that subtly introduces the inter- 
active responses to the viewer throughout the program. This 
enhanced interactivity is provided through the use of "trig- 
ger points" spread throughout the program. Trigger points 

10 occur at designated times and result in the program content 
being altered to present individual attention to the particular 
viewer. 

However, what is needed is an interactive personalization 
provided via an interactive multimedia computer. 

15 Furthermore, a system is needed that provides not only the 
ability to branch amongst parallel transmitted datastreams, 
but also, the capability to seamlessly integrate input from 
other media, such as CD-ROMs and laser disks, into the 
presentation. What is needed is a computer-based system for 

20 branching between a variety of inputs during the same 
interactive session including full-motion video, computer 
graphics, digital video overlays and audio. 

SUMMARY OF THE INVENTION 

25 The ACTV system is based upon branches which occur in 
the course of the full -motion video. Branches may be to 
other full-motion video segments, to graphics which are 
integrated into the video, and/or to audio segments which are 
integrated into the show. 

Sometimes, the ACTV system will act upon the user's 
response immediately; other times, it will utilize ACTV's 
unique "trigger point" concept to act upon the response later. 
ACTV's technology enables the computer to "remember" 

35 the user's responses and integrate them into the video and 
audio at a later point. Regardless of whether the action is 
taken as a result of the user's response immediately or later, 
it is done seamlessly. 
ACTV's television technology provides the capability to 

40 seamlessly branch among multiple video and audio sources. 
ACTV's computer technology provides the ability to seam- 
lessly branch not only among the multiple video and audio 
channels, but also to seamlessly integrate input from other 
media, such as CD-ROM's, laser disks, hard disks, and 

45 remote servers, connected via the Internet or another 
network, into the show. 

During a television-based ACTV interactive session, the 
system will branch among either multiple television chan- 
nels or multiple audio sources, depending upon the type of 

50 implementation. By contrast, during a computer-based inter- 
active session, branches may be among a variety of inputs 
from a variety of different sources during the same interac- 
tive session: full -motion video, computer graphics and 
audio. Since the computer provides the capability to process 

55 data from various multimedia inputs simultaneously, ACTV 
technology can integrate seamless switching of full-motion 
video, graphics and audio from various sources simulta- 
neously during the show. The computer-based ACTV imple- 
mentation is therefore much more flexible than the 

60 television-based ACTV implementation. 

It also provides the user with the capability to interact 
with the show utilizing a variety of input devices. Not only 
can the user interact with the show by pressing a multiple - 
choice button, but the interaction can also take the form of 

65 entry via the range of multi-sensory devices available on the 
computer, including mouse entry, full-motion pen entry and 
touch screens. This integration of various input and storage 
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devices is particularly valuable in an educational configuration for demultiplexing the digital audio stream 

environment, since it provides students with the ability to into n parallel digital audio channels, wherein the n parallel 

participate in their lessons in a variety of ways. The com- digital audio channels are time division multiplexed at the 

putercan both store the interactions for future reference and head-end and transmitted as a separate digital audio stream, 

also transmit them to the teacher, via either a computer 5 

network or, in a distance learning setting, via a telephone PREFERRED EMBODIMENT 

network. ^ snown in FIG. 1, the present invention is a computer 

An ACTV interactive session can integrate full-motion based system for receiving a fully interactive program, 

video with user input at the same time. For example, the allowing subscribers to interact with the program through 

full-motion video may be playing on the screen, while the 10 the use of a keypad and personal computer. Alternatively, the 

user is drawing a diagram in a corner of the screen. Thus, the multiple video/audio datastreams may be received from a 

video and audio may provide real-time input which the user broadcast transmission source or may be resident in local or 

is applying during the session on the same computer moni- external storage including CD ROM, video datatape, etc., as 

tor. discussed below. 

35 The interactive computer, 6 uses an interactive program 

DESCRIPTION OF THE DRAWINGS delivery system with any transmission means including 

FIG. 1 is a diagram of an interactive computer satellite, cable, wire or television broadcast to deliver the 

workstation, receiving inputs from television broadcasts interactive program (hereinafter "composite interactive 

and/or local storage devices. 20 program") from a centralized location, or operations center, 

rij o • a- p **- * y 4 for distribution to subscribers in their homes. The program 

FIG. 2 is a diagram of an interactive computer worksta- . . j . i- ^ i • ^ 

, . . 4 . t . t c . i • • may be broadcast live from the operations center. For 

Uon which receives its mput primarily from television J . . ^ . , F ,, , . , 

broadcasts example, live sporting events with added interactive ele- 
ments can be broadcast from the operations center. Such live 

FIG. 3 is a diagram of an interactive computer worksta- mte ractive elements could be different camera angles, slow 
Uon which receives its interactive programs entirely from 25 motioQ videQj etc Miemaiive{ y 9 the program can be pro- 
local storage, rather than television broadcasts. duced off _ Hne and stored in a program storage means at the 

FIG. 4 is a diagram of an interactive network for inter- operations center. Furthermore, the program can be pro- 
active processing. duced and stored locally at the remote site on CD ROM or 

FIG. 5 is a diagram of an interactive computer system, some other trans ferrable storage device such as digital or 

receiving inputs from a multichannel cable transmission and audio videotape, or laser disk. 

showing outputs via a conventional television monitor. An interactive presentation can comprise branching 

FIG. 6 is a block diagram of one interactive computer amongst full motion video, computer graphics and audio, 

workstation embodiment to achieve seamless switching with the interactive elements either received over a trans- 

between video signals. 35 mission media or stored locally, or both, all within the same 

FIG. 7 is a block diagram showing an alternative inter- show - ^ shown in mG - lhe workstation can branch 

active computer work station embodiment to achieve seam- amon S Vlde0 segments from television broadcasts, local 

less switching between video signals. Vlde ° s*™™ 38 ' 42 ( such 35 CD-ROMs, laser disks and tape 

i-t^ o ■ i_i i j- i_ . j . players), still images and audio segments from the preceding 

FIG. 8 is a block diagram showing another alternative to j- n 7t_ * jj--?m , , j. , -j j 

! ?• l _ *j • i ,o media, as well as those stored digitally on hard disks 34, and 

achieve seamless switching between video signals. 40 ' . P J , . y 

& & segments obtained over networks such as the Internet. 

FIG. 9 is a time diagram showing a representation of ^ mventi as shown ^ FIG ± ^ a for 

trigger points and corresponding alternative audio, video or ssi on a uter a ^ ^artive program allow- 

graphics segments, one of which is selected for presentation m usefS tQ ^ ^ thro fa a ter 

to the subscriber immediately after the execution of a trigger ^ ^ deyice 22 coaMtiM t0 a standard computer system 6> 

point tunction. . comprising a CPU 108 , hard disk 34> audio card ^ and 

FIG. 10 is a diagram of an interactive computer work monitor 18. The interactive multimedia computer 6 resides 

station embodiment for branching amongst multiple audio j n t he home of the subscriber or elsewhere, such as at a cable 

segments in a single video channel embodiment, where the headend, as described below. If at the home, the interactive 

interactive audio and data elements are embedded in the 5Q computer, 6 is usually located near the subscribers' 

video channel. television, if connected to the television set. Preferably, any 

FIG. 11 is a diagram of a second interactive computer of the multimedia computer embodiments, discussed below, 
work station embodiment for branching amongst multiple comprise a video demodulator board, a keypad for entering 
audio segments in a single video channel embodiment, subscriber selections, a data extractor board 46 (for extract- 
where the interactive audio segments are sent in the SAP 55 ing data from the vertical blanking interval from the video 
audio channel. signal(s), temporary and permanent data storage, a modem 

FIG. 12 is a diagram of a third interactive computer work 14, and a processor 108. 

station embodiment for branching amongst multiple audio Broadcast television is received by the video selector 10, 

segments in a single video channel embodiment, where two which selects among various television channels to capture 

tuners are employed; the first tuner for tuning to and $ 0 a video signal to be displayed on the computer monitor 18. 

demodulating the standard video and audio signal and the Multiple television channels may be received. FIG. 2 shows 

second of which is for demodulating a secondary analog an interactive computer workstation configuration which 

carrier comprising modulated serial digital audio segments. receives its input primarily from television broadcasts. With 

FIG. 13 is a diagram of a fourth interactive computer the technology of the present invention, seamless branching 

work station embodiment for branching amongst multiple 65 > s provided among these television channels, 

audio segments in a single video channel embodiment, also In the preferred embodiment, interactive programming is 

comprising two tuners, but with a digital demultiplexer also stored on Video Source devices 38, 42, as shown in 



03/18/2004, EAST Version: 1.4.1 



5,861,881 

5 6 

FIGS. 1 and 3. The Video Sources 38, 42 may be any local transmit the multimedia input to other students and teachers, 

storage device which is accessible by the computer, includ- via computer network and/or television broadcast. 

ing CD-ROMs, laser disks, VCR's and tape players. While As shown in FIG. 4, the interactive system of the present 

FIGS. 1 and 3 show only two video sources, there may be invention may operate on a computer network. In this 

any number of such devices. 5 configuration, the program is processed by the Video Server 

When CD-ROM 54 is employed in the present invention, 70 * ^ programs are sent over the network to the Client 

it is a component of a unique interactive experience. The Stations 58JS2, 66. Any number of client stations may be 

present invention utilizes CD-ROM 54 as one of the mul- supported. The configuration of each client station is pref- 

tiple input devices. Since branching is always seamless in erably lhe mteractive workstation as shown m FIG. 3. 

the preferred embodiment, the computer 6 may receive input 30 The control for integrating the various multimedia ele- 

from at least two devices, regardless of whether these ments 15 P rovided b y the A CTV authoring language, a 

sources are random access. This is necessary to avoid delays umc J ue set of interactive commands to facilitate the interac- 

during search periods. While one device is playing the video, tlve P rocess - commands may either be embedded into 

the other searches for a new branch. When the second device data P 0rtl0ns of full-motion video segments or may reside 

finds the segment for output display, the other input device 15 separately on a storage medium such as a Winchester disk. 

searches for a new branch. When the second device finds the When the w™™^ *™ embedded within the full-motion 

segment to be shown, the branch occurs seamlessly. The vldeo ( for exam P le * Wlthm the vertical blanking interval), 

apparatus and method for seamless branching among van- the interactions occur as soon as the computer completes the 

ous video signals is described in the paragraphs below. recognition of a command group. When the commands are 

c — . r a. ■ . «• i u . i 20 stored separately from the video segments in a digital 

Segments or the interactive program may also be stored zu . \_ A . : - , . ... , « • 

on the computer's hard disk 34. The segments stored on the timmg ° f the ! r f XeCUtl0n 15 based U P OD 

hard disk, 34 are usually computer graphics, still images or ?° t mtS - . ™ ese ngger P° ints a ? Ume P oints at ^ the 

audio segments, which are integrated into the presentation. * teractl0ns are t0 occur > as ex P la "* d » ™re ^tail below. 

The format for storage on the hard disk 34 is digital. Any . The » ser f an view the interactive program either directly 

storage device may, however, store any combination of 25 using the television set 90 or via the computer 94 screen as 

full-motion video, still images, graphics and audio seg- shown in FIG. 5. FIG. 5 is a diagram of an interactive 

ments subscriber station, receiving inputs from a multichannel 

A , ■ i i .1. • . j cable transmission and showing outputs via either the com- 

As shown in FIGS. 1-3, the interactive commands are . nA ? , , , . . ftA 

. . j e u + ur> , r t , puter 94 screen or a conventional television 90 monitor, 

extracted from the program by the Command Extractor 46. 5-, U i u i u u • • j r.^ 
AU , a. J < * .,30 Cable channels can be shown in a window on the PC screen 

Alternatively, these commands may be stored on an auxil- , , . , 4 JT .i_- LJ - 

j u . < i j j • i using conventional demodulator cards. In this embodiment, 

lary storage device such as the hard disk 34. J~ , . . , • *u i r i j- i 

L, t a cable set top box receives the plurality of analog or digital 

The commands are processed by the computer's Central video/audio signals from the multichannel cable. The inter- 

Processing Unit (CPU) 108, shown in FIGS. 1-3. The active multimedia computer 94 also receives the video/audio 

computer may be an IBM Personal Computer (PC)- 35 signals from the mult i c hannel cable and extracts the data 

Compatible, an Apple computer or any other type of stan- codeSj preferably embedded in the vertical blanking interval 

dard computer workstation. of the video signal ( s)> ^ interactive computer 94 detector 

The CPU 108 determines what video to display and audio detects and extracts data codes embedded in the data stream, 

to play based upon the interactive commands which it These codes are preferably sent to RAM memory and 

receives. Based upon the commands, it plays the appropriate 40 interpreted by the main processor. Personalized audio and/or 

input from its input devices, which are the Video Selector video selection occurs by the main processor sending a 

10, Video Sources 38, 42 and Hard Disk 34. Audio is branching command to the cable set top box. The cable set 

received and processed by the Audio Card 30 which sends top box processor interprets the command and seamlessly 

audio to Speakers 26 and/or headphones 50 as shown in branches to the selected video. 

FIGS. 1-3. 45 | n tne embodiment of FIG. 5, the subscriber can receive 
The user interacts with the program through the Input typical conventional video analog programming from a 
Device 22. This device may be a customized keypad, a cable headend. Cable systems also may be used to convey 
standard computer keyboard, a mouse to "point and click" at digital data via a system such as the High-Speed Cable Data 
selections and also to draw pictures, a touch screen (enabling Service (HSCDS). In a digital system, the subscriber stations 
the user to make a selection by pointing at the screen), a 50 may receive programming from content servers or Internet 
pen-based input device (for selecting options or draw Protocol (IP) routers. Content servers are typically a corn- 
pictures), a voice recognition device or any other computer bination computer and data storage system that stores vari- 
input device well known in the art. Furthermore, multiple ous types of content from information source providers, 
input devices may be accommodated by the system. These providers might provide anything ranging from video 
Regardless of the type of input device 22, user inputs can 55 games, distance learning applications, interactive 
be utilized by the present invention immediately, or at a later applications, home shopping applications, online magazines 
time, to result in personalized graphics, video and/or audio and newspapers, databases, and typical network and cable 
presentation. For example, the present invention utilizes programming. The IP router, on the other hand, formats, 
"trigger points/' as described below, to enable subsequent switches, and controls the flow of digital data traffic between 
branches among multimedia segments during the show. 60 the cable network and either the public switched telephone 
Additionally, more substantive user input, such as pictures network (PSTN), the Internet, or commercial on-line infor- 
and text, may be integrated into the interactive presentation. mation services, such as CompuServe and America Online. 
These types of user input are particularly useful in computer- A headend modem modulates the digital data generated by 
aided learning applications, since they enable students to the IP router onto an analog carrier signal suitable for 
participate in lessons utilizing various media. The interac- 65 transmission downstream to the subscriber. A typical down- 
tive computer 6 provides the framework to easily integrate stream modulation scheme is 64 Quadrature Amplitude 
the student's multimedia input into the session and to Modulation (QAM). 
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Each downstream transmission reaches the subscriber's passes through a forward error correction circuit 104 into a 
house, shown in FIG. 5, preferably through a tap and drop digital demultiplexer 106. The demultiplexer 106 is con- 
cable. The cable modem 92 demodulates the analog carrier trolled by microprocessor 108 to provide a specific video 
and converts the data to a digital format readable by the signal out of a number of video signals which may be 
user's PC 94. Alternatively, the cable modem can be 5 located within the data stream on the demodulated broadcast 
replaced by an RF demodulator board in the PC, 94. channel. The demultiplexed video signal is then decom- 

The programming content associated with the present pressed and decoded by decompressor/decoder 110. The 

invention may reside on either a headend-based or a remote video signal is synchronized by a sync add circuit 150 and 

content server or one of the storage devices, discussed above a sync generator 140. The video signal is then buffered by a 

(either temporarily or permanently downloaded from the 10 video buffer 160. The buffered video signal is modulated by 

content server). Subscribers gain access to the interactive a modulator 170 into a NTSC compatible signal, 

programming on the server via an online menu. By using a video frame buffer 160 and delaying the 

In this digital embodiment, one channel of digitally- viewing of a given signal, enough time is allowed for the 
compressed video content would require about 1.5 Mbps to decompressor/decoder 110 to lock onto, decompress, con- 
deliver VCR-quality images to the PC 94, while four chan- 15 vert to analog, and wait for the resultant vertical interval of 
nels would require about 6 Mbps. Thus, the interactive a second video signal. For example, assume video signal A 
system of the present invention fits within one 6 MHz is currently being processed and transferred through the 
channel. At the subscriber station, the interactive seamless circuit shown in FIG. 6 and displayed. Based upon a user 
system could be implemented in one of the interactive selection, the microprocessor 108 directs the digital demul- 
multimedia computers, described below. 20 tiplexer 106 and RF demodulator 102 to switch to another 

Seamless Switching between Broadcast Multiple Video v j deo signal, video signal B. To accomplish this, the analog 

Streams video from the first digital video signal, video signal A, 

Preferably, the digital video signals are compressed ^^^^syK.jsMi^v^b^bu&tim. 

(preferably via MPEG 2 or any other compression scheme) „ This buffer 160 can hold the full video picture for "n 

and multiplexed onto a standard NTSC signal. The circuitry 25 number ° f frames ^h the signal is output to the 

in FIGS. 6-8 below could be implemented on a board and ^P 1 ^ la c r effect ' ^delayed video signal A is viewed n 

inserted into a standard personal computer (PC). A separate " umber of fames ater the signal has been received. When 

microprocessor on the interactive board is not necessary for ^ selecti | a dlff f reDt vldeo P atn b y means of P ressm 8 

this configuration since the standard multimedia PC proces- m a button °? ' a ke yP ad ^ ^ olhe \ m f ans ' *° micr °- 

sor performs the functions of the processor 108 shown in 30 processor 108 instructs the digital demultiplexer 106 to stop 

FIGS 6-8 decoding signal A and lock onto signal B to begin decoding 

/at -r _j ... ct . . signal B instead of signal A. 

FIGS. 6-8 show preferred embodiments of the interactive ° 

multimedia computer 6 of the present invention to enable While this * happening, even though the decompressor/ 

seamless flicker-free transparent switching between the digi- 35 decoder 110 » no lon S er decompressing video signal A, the 

tal video signals on the same channel or different channels. dis P la y * sti11 snowin S video ^ A because it is being 

"Seamless" means that the switch from one video signal to read from the buffer 160 M 8000 as decompressing and 

another is user imperceptible. These embodiments may be decoding occurs, the microprocessor 108 looks for the next 

connected to any transmission media or simply connected to vertical blanking interval (VBI) and instructs the video 

the output of any stand-alone storage means (such as CD 40 frame buffer 160 to switch t0 lts in P ut > rather than lts 

ROM) for the digitized multiplexed interactive program. buffered output at the occurrence of the VBI. 

Preferably, the interactive computer connects to a television Since the RF demodulator 102, forward error corrector 

or other display monitor. To provide this capability, only a 104, digital demultiplexer 106, and decompressor/decoder 

digital demultiplexer, decompressor(s), frame buffers), and HO require a certain time period to decompress and decode 

sync components are added to the conventional multimedia 45 me video signal B frame from its data stream, the size of the 

personal computer. These items, and any other components, buffer 160 has to be large enough so that this processing can 

may be connected to the PC processor and storage elements take place without interruption during the switching of the 

in the manner disclosed in FIGS. 6--8. video signals. If desired, the system may continue to use the 

FIG. 6 shows an embodiment which allows for a seamless buffer 160 in anticipation of a future switch. By using the 

video switch between two or more separate digital video 50 microprocessor 108 to manipulate the fill and empty rate of 

signals. As shown in FIG, 6, a CPU 108 is connected to an lhe buffer 160 > the buffer 160 ma y be ra P ic % filled with 

RF demodulator 102 and digital demultiplexer 106. The video si S nal B frames and then after a P eriod of time wiU be 

CPU 108 directs demodulation and demultiplexing of the reset and read y t0 make another switch t0 another vide0 ™ 

proper channel and data stream to obtain the correct video lhe same manner - The buffer 160 ma y also reset b y 

signal. Preferably, switches occur at an "I" frame if MPEG2 55 ski PP in S frames or providing a delay between sequential 

compression is used. The proper channel is determined frame out P uts for a shorl Ume in order 10 fiU the buffer - If a 

either by examination of the user's input from user interface dela y is to maintain video signal or frame output while 

130 and/or any other information or criteria (such as per- lhe buffer 160 15 bem S fillcd a distortion may occur for 

sonal profile information) stored in RAM/ROM 120. For a bnef amount of time - 

example, the RAM/ROM 120 could store commands pro- 6 o Because a first video signal is always displayed as the 

vided within the video signals as discussed in U.S. Pat. No. output of the buffer 160 after the delay, the buffered video 

4,602,279, and incorporated herein by reference. The user masks the acquisition and decoding of a second video signal, 

interface 130 may be an infrared, wireless, or wired receiver As long as the buffer 160 is large enough to keep the first 

that receives information from a user interface unit. video running while the second video is being decompressed 

The RF demodulator 102 is part of the receiver, and 65 and decoded, a seamless switch will occur, 

demodulates data from the broadcast channel directed by the FIG. 7 shows an alternate, dual tuner embodiment for 

processor 108. After the data stream is demodulated, it seamless switching between separate video signals. In this 
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embodiment, the microprocessor 108 controls the selection 
of the RF channel that is demodulated by RF demodulators 
102A, 102B. The demodulated data streams enter the for- 
ward error correctors 104A, 104B. At the output of the 
forward error correctors 104A, 104B, the data streams are 5 
transmitted to the input of the digital demultiplexers 106A, 
106B. 

As with the RF demodulators 102A, 102B, the digital 
demultiplexers 106A, 106B are controlled by the micropro- 
cessor 108. This configuration allows the microprocessor 10 
108 to independently select two different individual time- 
multiplexed video signals on different channels and data 
streams. If all the video signals of an interactive program 
were contained on a single channel or data stream, it would 
only be necessary to have a single RF demodulator, forward 15 
error corrector, and digital demultiplexer serially connected 
and feeding into the two digital video buffers 164, 165. 

Two data streams are provided from the digital demulti- 
plexers 106 A, 106B. One data stream carries video infor- 
mation pertaining to the video signal the user is currently 20 
viewing. The second data stream carries the video signal 
selected based on the user's previous and/or current inter- 
active selections from the user interface, as determined by 
the microprocessor 108. 

The digital information on each of the two streams is 25 
buffered in digital video buffers 164, 165. The buffered 
signals are then decompressed and converted into analog 
signals by decompressors/decoders 110A, HOB which 
include digital to analog converters. The decompressors 
110A, HOB are preferably MPEG2 decoders. 

A local sync generator 140 is connected to sync add 151, 
152 and frame sync circuits 153, 154. Because both streams 
are synchronized based on signals from the same local sync 
generator 140, each stream becomes synchronized to the 35 
other. In particular, the signals on each stream are frame 
synchronized. 

A vertical blanking interval (VBI) switch 180 is con- 
nected to the microprocessor 108 so that the input may be 
switched during the vertical blanking interval of the current 40 
stream, resulting in a seamless switch to the viewer. 

The embodiment of FIG. 7 operates as follows. Based on 
user responses and control codes, it is assumed that the 
microprocessor 108 determines that a switch from video 
signal A to video signal C should be performed. RF demodu- 45 
lator 102A and digital demultiplexer 106A are processing 
the currently viewed video signal, video signal A, which is 
progressing through the upper branch components. A com- 
mand is issued from the microprocessor 108 to the RF 
demodulator 102B commanding a switch to the channel and 50 
data stream on which video signal C is located. The micro- 
processor 108 also instructs the digital demultiplexer 106B 
to provide video signal C from the received data stream to 
digital video buffer 165. 

At this point, the upper RF demodulator 102A and digital 55 
demultiplexer 106A are still independently receiving and 
processing video signal A, which continues through the 
upper branch of the circuit. 

At a certain point, the digital decompressor/decoder HOB 
in the lower branch will begin filling up with video C frames. 60 
After the video signal C is decompressed and decoded, it is 
converted into analog. A local sync generator 140 inserts 
both local sync and frame sync to video signal C via sync 
add circuit 152 and frame sync circuit 154 in order to 
synchronize it with the currently displayed video signal A, 65 
which is still being provided from the upper digital video 
buffer 164. At the appropriate switch point, triggered by 
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programming codes supplied with each video signal A and 
C, the microprocessor 108 directs the VBI switch 180 to 
switch in the vertical blanking interval from video A to video 
C, at which time video C will then seamlessly appear on the 
computer screen. 

Digital video buffers 164, 165 may be used in the circuit 
of FIG. 7, but are optional. However, in an alternative 
embodiment the buffers, 164, 165 would be required to 
provide a seamless switch if the FIG. 7 circuit was modified 
to incorporate a single RF demodulator 102, single forward 
error corrector 104, and single digital demultiplexer 106 (as 
in FIG. 3), each with a single input and single output. In this 
alternative embodiment, the circuit cannot independently 
receive and demultiplex two data streams on different fre- 
quency channels. One buffer is used to store previously 
received video signals, while the other buffer quickly passes 
through the selected video signals. 

Based on the same assumptions above, video signal A is 
progressing through the upper branch of the circuit and it is 
desired to switch to video signal C. However, in this 
alternative embodiment, the digital video buffer 164 is 
providing maximum buffering to video signal A. 

Because it is desired to switch to video signal C, the 
microprocessor 108 directs the alternative circuit 
(containing a single RF receiver 102, single forward error 
corrector 104 and single digital demultiplexer 106 connected 
in serial), to receive and demultiplex the data stream on 
which video signal C is located, which may be different than 
that of video signal A. When video signal C is 
demultiplexed, the microprocessor 108 directs the digital 
video buffer 165 to provide minimum buffering of video 
signal C so that decompressor/decoder HOB may quickly 
decompress and decode the digital signals. After decom- 
pression and decoding, video signal C is synchronized with 
video signal A. At this time video signal A is read for display 
from digital video buffer 164. The upper digital video buffer 
164 must be large enough to provide video frames for output 
during the time it takes the RF demodulator and digital 
demultiplexer to switch to video signal C and the time 
required for decompression, decoding, and synchronization 
of video signal C. 

When video signal C is synchronized with video signal A, 
the microprocessor 108 directs VBI switch 180 to switch 
from video signal A to video signal C in the vertical blanking 
interval of video signal A, thereby providing a seamless and 
flicker-free switch. 

At this time, digital video buffer 165 will begin to utilize 
maximum buffering by altering its fill/empty rate as 
described above with respect to the FIG. 7 embodiment. 
When adequate buffering is achieved, a switch to another 
video signal may be performed in the same manner as 
described above. 

Another preferred embodiment is shown in FIG. 8. This 
embodiment also includes an RF demodulator 102, a for- 
ward error corrector 104, and a digital demultiplexer 106. 
However, the circuitry differs along the rest of the chain to 
the television set or monitor. In this embodiment, a memory 
190 is incorporated and connected to the output of the 
demultiplexer 106 for storing the compressed composite 
digital video signal. The decompressor/decoder 110 is 
inserted at the output of the compressed memory 190. The 
decompressor/decoder HO decompresses the digital signal, 
converts the signal to analog and forwards the analog signal 
to the RF encoder, 155 for transmission to the monitor. Once 
the composite compressed digital video signal is fed into the 
compressed memory 190, the microprocessor 108 directs a 
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pointer to be placed somewhere along the compressed 
digital video signal. Based on the placement of the pointer, 
different frames and different segments of the composite 
digital video signal will be read from memory 190 for 
decompression and decoding. 5 

The different video signals are distinguished from one 
another because they are labeled, preferably by headers. 
Assuming that video signal A has been selected for play on 
the monitor, the compressed digital memory 190 fills up with 
A frames. Assuming a switch to video signal C is desired, the 30 
microprocessor 108 directs the RF demodulator 102 and 
digital demultiplexer 106 to begin filling the compressed 
memory 190 with video C frames. The decoder pointer 
begins to move down. As soon as a sufficient number of C 
frames have entered the compressed memory, the pointer 35 
will then jump to the beginning of the C frames. The C 
frames are then output into the decompressor/decoder where 
the digital frames are converted into an analog signal. 

The digital video is multiplexed in a series of easily 
identifiable packets. These packets may contain full com- 20 
pressed frames of video (I frames) or may include only the 
differences between full frames (B frames or P frames). 

To be able to reconstruct the full video images, the 
decompressor/decoder 110 needs to have a minimum num- 25 
ber of I, P and B frames. The decoder 110 needs only one I 
frame to decode an image. Conversely, two prior Anchor 
frames ('Ts" and "P's") are necessary to decode B frames. 
In order to decode P frames, the decoder 110 only needs one 
Prior Anchor frame. When the microprocessor 108 instructs 3Q 
the digital demux 106 to start sending packets from a 
different data stream there is no way to be certain that the 
next packet will be an I packet needed for decoding the 
second video stream. To avoid a breakup of the video 
images, which would occur if the decompressor/decoder 110 35 
suddenly started receiving packets unrelated to the stream it 
was decoding, the microprocessor 108 starts to fill up the 
memory 190 with video signal C packets until it is deter- 
mined that a full sequence of I, B and P frames are available. 
The decoder, 110 should receive the last bit of the last B 4Q 
frame in a given, GOP (Group of Pictures) before the switch, 
in order to prevent glitches when decoding. Furthermore, the 
last B frame of the GOP must only be backward predicted, 
not forward predicted or bidirectional predicted. As soon as 
the valid sequence is in memory 190 the microprocessor 108 45 
moves the memory read pointer to the start of a valid 
sequence of C video signal packets so that the 
decompressor/decoder 110 can successfully decode the C 
signals. This results in a seamless switch from video signal 
A to video signal C. 5Q 

This embodiment requires a data channel for enabling a 
synchronous switch between a first video stream and a 
second video stream. This data channel comprises the ACTV 
codes which link together the different program elements 
and information segments on the different video signals. In 55 
addition, the data channel also comprises synchronization 
pulses and a time code to signify to the pointer the proper 
time to skip from a memory location representing one video 
signal to a memory location representing another video 
signal in order to enable a seamless switch. 60 

The microprocessor 108 reads the data signal from the 
digital demultiplexer 106 and communicates pertinent data 
to the sync add circuit 150, which is connected to sync 
generator 140. The microprocessor 108 is then able to 
synchronously communicate with the memory 190. 65 

r I ne time code sent will identify the timing for one picture, 
as well as for multiple pictures, and will lock the different 
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pictures together. This is done through the use of similar 
docks at both the transmission end and the receiver. A time 
code is used in order to keep the two clocks at both the 
transmission and receive end synchronously connected to 
one another. Once the clocks at both ends are working 
synchronously, each of the multiplexed video streams must 
be synchronized to the clocks. In order to synchronize the 
multiplexed video stream to the clocks, each of the indi- 
vidual channels must be referenced to a common reference 
point and must be identified. 

In the preferred embodiment, a packet header would be 
incorporated into the transport layer of the MPEG signal to 
identify the various channels. The packet header will also 
include information as to where to insert the vertical blank- 
ing interval. In MPEG, the vertical blanking interval is not 
transmitted from the headend. Therefore, the vertical blank- 
ing interval must be generated locally. The packet header 
eye will identify at what time the vertical blanking interval 
is in existence in order to effectuate a seamless switch 
between analog pictures. 

In summary, the combination of clock and the information 
imbedded in either the transport layer of MPEG or in a 
separate packet on a separate data channel effectuates the 
linking between each video signal and a corresponding time 
point. The data channel also includes information designat- 
ing when all the various video signals will be in synchronism 
with one another. It is at these points that the microprocessor 
108 may direct the pointer to skip from one location to 
another location, at a time (such as during the VBI) when a 
seamless switch will result. 
Trigger Points 

Interactivity is further enhanced in the interactive com- 
puter workstation embodiments through the application of 
trigger points 900 scattered at various predetermined times 
throughout the program, a timeline representation of which 
is shown in FIG. 9. The trigger points 900 correspond to 
times when interactive events are scheduled to take place. 
These interactive events could be the selection and playing 
of video, audio segments or the display of graphics. While 
the choice of particular video, audio or graphics is still 
dependent on viewer selections, the viewer selections in 
response to displayed graphical interrogatory messages are 
preferably made during a period at the onset of the program 
or when a viewer first tunes into the program. These viewer 
selections are then utilized as inputs to macros called up at 
later times during the program by the controller upon the 
occurrence of the trigger points, identified to the interactive 
computer by unique codes embedded in the video signal. 

The trigger points correspond to the times when the 
conventional program content can be altered and personal- 
ized for those subscribers capable of receiving the interac- 
tive signal. The programmer can place the trigger points at 
any time throughout the program. Since the trigger points 
are unknown to the subscriber, the subscriber docs not know 
when they will receive a personalized message. In other 
words, an interactive response can either immediately fol- 
low a corresponding user selection made to an interrogatory 
message or occur at a later time corresponding to a trigger 
point, or any combination of the two. Of course, timing of 
the interactive events should correspond to suitable times in 
the program where branching to interactive elements is 
sensible and does not clash with the program content of the 
conventional video still displayed on the television or other 
display monitor. 

At the onset of a trigger point 900, the controller will 
select one of several possible audio (or video or graphic 
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display) responses for presentation to the subscriber. As 
mentioned above and shown in FIG. 9, some of the 
responses may comprise a branch to either a video segment 
and/or audio segments. 

In combination with the use of trigger points 900, the 
present invention allows for the viewer to select certain 
options at the onset of the program to suit the viewers' 
preferences. For example, if the program broadcast is a live 
sports event, at an early trigger point 900, the viewer could 
be queried as to whether the viewer would prefer to receive 
audio in English, Spanish, French, or perhaps hear the local 
announcer instead of the network announcer. Upon the 
viewer selection, the CPU directs a branch to the appropriate 
interactive segment. 

Each trigger point is identified preferably through the 
broadcast of ACTV codes sent as part of the composite 
interactive program signal. The codes preferably include, at 
a minimum, the following information: (1) header identify- 
ing the occurrence of a trigger point; (2) function ID (e.g., 
selection of audio or graphics responses, etc.); and (3) 
corresponding interrogatory message(s). The first bit 
sequence simply identifies to the controller that a trigger 
point is about to occur. The function ID designates the macro 
or other set of executable instructions for the controller to 
read and interpret to obtain the desired result, e.g., a selected 
video and/or audio response. 

Upon extraction of the codes by the data decoder, the CPU 
108 reads and interprets the codes and calls from memory a 
particular user selection(s) designated by the trigger point 
codes. The user selections correspond to subscriber answers 
to a series of interrogatory messages preferably presented at 
the beginning of the program. After obtaining the appropri- 
ate user selections), the controller 108 reads and performs 
the executable instructions using the user selection(s) as 
input(s) in the macro algorithm. The result of the algorithm 
is either a selected video stream, audio and/or selected 
graphics response. The video/audio response can be called 
from memory if it is prestored, called from external data 
storage, or the controller can command the switch to branch 
to the particular video audio stream if the response is 
broadcast concurrently with the trigger point. After the 
selected video/audio response is played to the subscriber, the 
switch branches back to the standard program, shown at time 
t, in FIG. 9. 

As mentioned above, a series of interrogatory messages 
are preferably presented when the subscriber begins watch- 
ing the interactive program. These interrogatory messages 
can be presented in any one of three ways. First, the 
interrogatory messages can be presented as graphics dis- 
plays overlaid by the interactive computer workstation onto 
a video signal, wherein the graphics data is sent in the 
vertical blanking interval of the composite interactive signal, 
or alternatively stored on the hard disk or external storage. 
Second, the interrogatory messages are presented as graph- 
ics displays as discussed above, except the graphics data 
comes from local storage, external data storage (e.g., CD 
ROM, cartridge, etc.), or a combination of data in the VBI 
and data called from either local or external data storage. 
Third, graphics data can be presented in the form of user 
templates stored at the interactive computer workstation. 

User selections corresponding to answers to the n suc- 
cessive interrogatory messages are received by the remote 
interface at the beginning of the show, stored in memory and 
used throughout the show at the appropriate trigger points to 
subtlety change program content as the show progresses. 
Preferably, each interrogatory has a set of possible answers. 
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Next to each possible answer will be some identifier corre- 
sponding to a label on a key on the user interface. The 
subscriber depresses the key corresponding to their answer 
selection. This selection is decoded by the remove interface 
and controller, stored in memory, preferably RAM, and used 
later as required by an algorithm designated at a trigger 
point. 

Single Video Channel Interactive Computer Embodi- 
ments Providing Personalized Audio Responses 

While such interactive programming may include a plu- 
rality of video signals, the interactive multimedia computer 
work station 6, described herein, may also provide for 
personalized audio interactivity by way of a single standard 
video and audio television signal with a plurality of addi- 
tional audio signals and/or graphics data for providing 
interactivity, as shown in FIGS. 10-13. The interaction with 
the subscribers comes primarily by way of selection of one 
or more linked audio segments from a plurality of audio 
segments, whereby the selected audio segment(s) are chosen 
as a function of previous user responses. Interactivity is 
enhanced through the use of overlaid graphics displays on 
the video which like the audio responses, also vary accord- 
ing to selections made by the subscriber on the user inter- 
face. Audio segments are used to provide personalized 
responses to subscriber selections. The graphics, on the 
other hand, are used to both query the subscriber, preferably 
at the beginning of the program, and also to provide per- 
sonalized graphical messages to subscribers. The interactive 
show also comprises control data for controlling the inter- 
active computer work station. 

Multiple audio segments forming the set of suitable 
responses to an interrogatory message can be sent as part of 
a standard video signal. There are a number of different ways 
to effectively forward the necessary audio segments for a 
given interactive event to the interactive computer. The 
interactive elements may be broadcast synchronously 
(alternative responses aligned in time), serially, on separate 
channels, embedded in the existing video and/or transmitted 
before or during the program. Audio segments tagged for a 
given interactive event, can be sent to the interactive com- 
puter work stations much earlier than the scheduled event 
during the program, in which case the segments are prefer- 
ably stored in temporary memory, or the segments can be 
transmitted concurrently with the event. With the present 
invention, it makes no difference how the audio segments 
reach the interactive computer work station as long as they 
are available for selection at the computer 6 at the prede- 
termined "trigger points," described below. For example, the 
audio segments could also be stored in local external data 
storage such as CD-ROM, 

In one preferred "trigger point" embodiment, interactive 
audio shows can be delivered in the standard television 
channel. In this embodiment, four audio responses are 
available at each trigger point, however, only two audio 
channels need be broadcast, or otherwise input, to the 
interactive computer 6. 

This embodiment has the advantage of requiring merely 
one television channel. Channel 1 is the "home" channel. 
When channel 1 is playing, channel 2 is used to download 
the audio for tracks 3 and 4 to the interactive computer 6. 
This downloaded audio is stored as wave files in the local 
unit. When it is time to branch, audio tracks 1 and 2 are 
played on the two audio input channels, while tracks 3 and 
4 are generated from the audio wave files on the interactive 
computer 6, A seamless branch is made from any one of 
these channels to any of the other channels. 
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FIG. 10 shows an overview of a preferred interactive 
computer work station embodiment. Other digital and audio 
alternative embodiments for the provision of audio interac- 
tivity are shown in FIGS. 6-8 of U.S. patent application Ser. 
No. 08/289,499, herein incorporated by reference. The 
embodiments represent different apparatus for receiving, 
processing and storing the alternative interactive audio seg- 
ments which are received in different transmission formats. 
With these embodiments, the interactive systems are no 
longer solely limited to selecting audio from multiple par- 
allel tracks of audio, related in time and content, nor is the 
interactive questions-immediate answer format, as disclosed 
in previous patents, necessary. Of course, the systems of the 
present invention can still use the question-immediate 
answer format or a combination of such format and delayed 
response via trigger points. The concept remains the same, 
i.e., to select audio responses which are matched to user 
selections by some function. 

The elements of the audio interactive embodiment can be 
incorporated and provided by the interactive multi-media 
work station. Preferably, this configuration comprises a 
video demodulator board, a keypad for entering subscriber 
selections, an extractor board for separating the audio sig- 
nals and data from the conventional video signal, temporary 
and permanent data storage, a modem 312 (optional), audio 
switch 620 and a processor 178. 

Referring to a preferred embodiment shown in FIG. 10, 
the video demodulator 616 outputs the standard video signal 
which is transported to a Gen-lock circuit 623 and character 
generator 624 as well as to a voice/data extractor 174. At the 
output of the Gen-Lock circuit 623 and character generator 
624, the video is forwarded via the RF modulator 622 to the 
television or computer display monitor. The processor 178 
preferably controls an nxl switch 620, the output of which 
is an appropriate audio segment to be sent to the television 
set for presentation to the subscriber. Of course, the switch 
could have more than one output, in which case more than 
one viewer can watch the video on the same monitor and 
each receives individualized audio response through the use 
of headphones. The processor 178 sends a command to the 
audio switch 620 to disconnect the standard audio at the 
beginning of an interactive segment. The extractor,174 
essentially reverses the process by which the audio and data 
signals were inserted into the video signal. As explained 
below, the voice/data extractor 174 removes the additional 
audio segments and data that are hidden in the standard 
video signal. The data is forwarded to the microprocessor 
178 and the audio segments get sent either to an audio switch 
620 or to temporary memory 202 depending on where the 
instructions teach the segments to be forwarded, all of which 
occurs under the control of the microprocessor 178. The 
microprocessor 178 reads and interprets the instructions 
either broadcast in the data codes or resident in the operating 
software at the interactive work station 6. 

The microprocessor 178 interprets the extracted data as 
either control data, including instructions for switching 
between voice channels, or graphics data for on screen 
display. If the data is on-screen display data, the data is 
preferably prefixed by a command designating the data as 
on-screen display data, as opposed to control data. In the 
preferred embodiment, the controller 178 also examines the 
control data for the occurrence of a header code designating 
the onset of a trigger point in the program. 

If the trigger point codes designate a macro which calls 
for the placement of a graphics display on the video, the 
microprocessor 178 reads the codes, accepts any graphics 
data sent from the head-end, calls for and examines the 
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actual bit maps stored in memory 282, 284, 286 or external 
memory 629 and designating the identity of the characters, 
and then commands the character generator 624 to overlay 
particular characters at particular points on the screen. 

5 Therefore, the graphics are preferably generated locally with 
the bit maps stored in memory 289. The graphics are 
selected for presentation either in predetermined sequence, 
through the use of control codes in the composite interactive 
program, developed when the program was created at the 

a0 operations center, or more flexibly through the execution of 
algorithms by the processor 178 utilizing stored subscriber 
selections to previous graphic interrogatory messages. The 
algorithms are preferably part of the operating systems 
software stored in memory at the interactive work station. 

15 Alternatively, the algorithms could be included in the data 
portion of the composite interactive signal. 

The graphics can be utilized to overlay any portion of the 
screen of the television screen. The character generator 624 
is locked by a Gen-lock circuit 623 which allows for the 

20 synchronous placement of the graphics on the video. The 
character generator 624 is preferably a standard on-screen 
display chip which takes incoming video, locks the video 
and superimposes on the video the characters as instructed 
by the microprocessor 178. Specifically, the character gen- 

25 erator 624 is a switching system which takes the active lines 
of video and switches to a mode of sending the graphics 
characters for a predetermined time, and then switches back 
to the video when the character is finished being written on 
the screen. 

30 Because the graphics are generated locally, subscribers 
without the interactive multimedia computer 6 are not be 
able to view the graphics. For those subscribers possessing 
the interactive capability, the graphics can be used for both 
posing interrogatory questions to subscribers at the onset of 

35 the program, consistent with the trigger point embodiment, 
posing questions during the program, or used to provide a 
personalized response to previous individual subscriber 
selections. 

Preferably at the beginning of the program or when a 

40 viewer first tunes in, a series of interrogatory messages are 
presented to the subscriber. The subscriber responds to the 
interrogatory message by depressing a button via the user 
interface device corresponding to an answer selection listed 
on the interrogatory graphics screen. If the subscriber has 

45 made a selection using a remote, a signal is received by the 
IR interface 628 which processes the signal and forwards the 
signal to the processor 178. The processor preferably creates 
a packet comprising the user selection and a header code that 
identifies the particular interrogatory message associated 

50 with user selection and sends the packet to memory 284. 
Each user selection to each interrogatory is stored in this 
fashion. These selections will be called later in the program 
at appropriate times when identified by the trigger point 
codes and then used in macros or algorithms to determine 

55 interactive audio and/or graphics responses. 

The presentation of the graphics interrogatory messages 
can also be made a function of subscriber selections to 
previous interrogatory messages. The logic used in the codes 
for selecting the next graphics message is similar to that used 

60 for selecting audio messages. One method, as disclosed in 
earlier ACTV patents, is the "decision tree" logic method- 
ology. The subscriber makes a selection to a first predeter- 
mined interrogatory graphics message. After the subscriber 
hears an appropriately branched audio channel, the proces- 

65 sor 178 will interpret graphics algorithmic codes sent down 
from the operations center 608 and will read from memory 
284 an appropriate next graphics message. The processor 
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178 then directs the character generator 624 to overlay the The RF demodulator 616 again demodulates the conven- 

seiected graphics message onto the next frames of video. tional video and audio signal. The data extractor 175 

The advantages discussed above in relation to presenting receives the signal and extracts the VBI line 21 data codes, 

an interactive program using trigger points are obtainable in ™ e <?ata in the VBI indicates the frequency channels in 

each of the interactive computer embodiments shown in 5 which the digital audio segments are transmitted For 

cr^o 11 11 t *l i j ■ , « . rir > I* lf „ example, audio messages A-E are located in between chan- 

MGS. 11-13. In the embodiment shown in Flu. 11, alter- i n i ie t*. . « • . , «i_ j * * 

- . „ f 4| _ nets 14 and 15. The controller 17s instructs the data tuner 

native audio segment are preferably sent serially from the 615 tQ ^ (o ^ q[ ^ ^ betw£en channels „ 

operations center in the SAP channel. The demodulator 617 and 15 Mem1ivcl an autotune capabil i ty can be used to 

receives a composite interactive signal comprising the stan- find me audio channels m the spectTllm , 

dard video and standard audio signal along with an audio 10 ^ ^ ^ demodulates ^ # ^ audio gi al and 

subcarner. The demodulator 617 breaks the signal into it s forwards ^ si j tQ ^ demultiplexer 700 

component parts forwarduig the baseband video to a data demuUi lexer 700 d^uitip^es the s i gna i int o n digital 

extractor 175 and the standard audio to an audio switch 620. audjo cha[mels and forwards each channe , tQ a te D/A 

The line 21 data extractor 175 takes out the data codes, converter 702 _ 710 where each of the digital channels are 

including the trigger points. » convefted 

to analog audio. As described above, one of these 

The SAP channel comprises a plurality of audio segments channels 712 can be selected as identified at the trigger 

lined up serially. The audio segments are digitized in the points for play over the audio speaker to the subscriber, 

analog to digital converter 750 and are preferably stored in ^ embodiments described above and shown in connec- 

digital audio memory 283. At certain times during the tion with FIGS. 10-13 relate to different ways of receiving 

program, data codes will designate a trigger point and key broadcast aud i 0 segments. Alternatively, interactive audio 

the microprocessor 178 to select and play an audio segment segments, or graphics elements, could be prestored on 

corresponding to previous user inputs), according to the cartridge, CD ROM, an audio card, or even floppy disk, 

process described above. The microprocessor 178 calls the EyeQ m0fe eflhaaced &nd ^ caQ occuf 

appropriate audio segment(s) from internal memory or m h the additkm of external data st such ^ CD 

external data storage 629 and commands the audio switch to ROM or cartridge. For example, sports statistics or other 

pass the selected audio segment to the RP modulator 622 for informalion on athletes or others can be stored in CD ROM . 

play to the subscriber At the end of the interactive time Duri ^ eyeQt ^ ^ nis or hics 

period, the controller 178 instructs the audio switch 620 to ^ focusi Qn me caQ be ^ b ^ 

again pick up the standard audio. ^ cessof and presented t0 the viewer as a fo nclion of user 

In an alternative embodiment similar to that as shown in selection of an option or at a trigger point if the user 

FIG. 11 and discussed above, the simple addition of a second indicated during queries at the beginning of the live event 

tuner, receiving the composite RF signal, could be used to that they were interested in a particular player, 

tune to a second audio channel for collection of transmitted Memory 

audio segments. The tuner would pass the audio segments to 3J ^ mteractive mmpulel a]s0 has the advantage of 

the A/D converter with the operation of the rest of the rememberi subscriber responses and using these 

interactive computer workstation similar to that described DXS in choosi a video/audio resp onse, and/or graph- 

above m connection with FIG. 11. * • . . . f u . , . , , 

ics interrogatory message, to present to the student. Memory 

FIG. 12 shows another interactive computer workstation branching is a technique of the present invention where the 
embodiment for providing alternative audio and graphics 40 algorithm assembles video/audio responses and graphics 
segments. This embodiment uses two tuners: an RF demodu- interrogatory messages according to the current and previ- 
lator 616 and a data tuner 615. The RF demodulator 616 ous use r inputs. Memory branching is accomplished by 
tunes to and demodulates the conventional video and audio linking video/audio streams and/or successive graphics 
signal in the standard video bandwidth. The data tuner 615 interrogatory messages together in a logical relationship, as 
receives a single digital audio signal. The signal comprises 45 described in U.S. application Ser. No. 08/228,355, herein 
digital serial audio segments modulated onto an analog incorporated by references. In this scheme, the interactive 
carrier. The data tuner 615 demodulates the signal into computer processor contains logic (preferably, in the soft- 
digital audio. The digital interface selector and error correc- ware algorithm) and memory to store previous subscriber 
tor 177 separates the audio segments and performs error selections and to process these previous responses in the 
correction according to any error correction scheme com- 50 algorithm to control future video/audio stream selection, as 
monly understood in the art. The controller 178 directs the we u ^ future graphics message selection, 
selector 177 to extract selected digital audio segments from u ser Profiles 

the serial digital stream and send them to the digital audio In a preferred embodiment, the interactive computer can 

memory 283. Selection of one or more audio segments for have stored in its memory a "user profile." The "user profile" 

play as personalized messages on the speakers occurs 55 preferably contains characteristics of the particular viewer at 

according to the processes described above. After the con- that subscriber location, such as sex, hobbies, interests, etc. 

trollcr 178 commands the memory 283 to forward a digital This user profile is created by having the user respond to a 

audio segment, the segment is converted to analog by the szrks of questions. Alternatively, the user profiles could be 

digital to audio converter 176 and is subsequently passed to created at a host and sent to the interactive computer over a 

the RF modulator 622 for play on the speakers. 60 network. This information is then used by the interactive 

Another interactive computer 6 workstation embodiment computer software to create a compendium of the viewer's 

for receiving, storing and selecting alternative audio seg- interests and preferences — i.e., a user profile. The stored 

ments is shown in FIG. 13. At the operations center, the user profile would be used in place of the question/answer 

audio segments are digitized, time division multiplexed, format, and thus, dictate the branches to interactive seg- 

modulated and converted to frequencies in unused channel 65 ments of interest to the viewer. 

frequency space in the cable television spectrum, e.g., cable Alternatively, the interactive computer 6 can be pro- 
guard bands. grammed to create a user profile of each viewer based on the 
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selections made during one of the interactive programs. 
Furthermore, such a user profile could be modified or 
enriched over time based on selections made during future 
interactive programs. For example, the * memory* technique 
described above can be used to modify the user profile based 5 
on user response over time. 

Once the profile is created, the programming choices or 
interactive responses can be triggered based on the content 
of the user profile itself. For example, if the user profile 
suggests that the viewer is particularly interested in sports 10 
cars, a sports car commercial could be played for the viewer 
at a predetermined point in the program. As another 
application, if a viewer's user profile indicates that the 
viewer is interested in cooking, whenever the viewer 
watches such a program, the user profile would trigger the 15 
interactive program to download recipes and either display 
such recipes on the screen or send the recipes to an attached 
printer 302. 

Applications 

The embodiments, described above, allow for several 20 
possible applications. For example, in a live sports event, 
one channel could carry the standard video channel, with 
other channels carrying different camera angles and/or 
close-ups of particular players. 

Audio interactive applications include the recording of 25 
audio clips for each player in the game. In this application, 
the viewer may access a pull-down menu, where he can 
choose a name of a particular player in the game. When this 
selection is made, the appropriate audio segment is called 
from memory and played for the viewer. In a similar manner, 30 
statistics in the form of text and graphics can be displayed 
for a selected player. 

Internet Applications 

Interactive programs of the present invention can be 
created using the Internet. Interactive program authors can 35 
access a particular Internet site and download graphics, 
audio and video clips and suggested interactions. The author 
can then use these elements in the authoring tools to create 
an interactive program. 

Furthermore, viewers can watch interactive programs 40 
from the Internet itself using the systems of the present 
invention. From an Internet site, viewers can access a single 
channel interactive program, such as described above. The 
viewer would watch the video on his or her computer, while 
the audio and/or text/graphics from Web site locations, for 45 
example,would be presented as a function of his or her 
specific choices via interactive commands. 

In addition, viewers can choose between multiple video 
streams originating from a site on the Internet. The seamless 
branching between different video streams would occur 50 
through interactive commands resident in the viewer's com- 
puter. 

Using the foregoing embodiments, methods and 
processes, the interactive multimedia computer maximizes 
personalized attention and interactivity to subscribers in 5s 
their homes in real time. Although the present invention has 
been described in detail with respect to certain embodiments 
and examples, variations and modifications exist which are 
within the scope of the present invention as defined in the 
following claims. 60 

We claim: 

1. An interactive computer work station for presenting an 
integrated multimedia interactive presentation, comprising: 

a means for receiving an integrated audio, graphics and 
video presentation, the presentation comprising a com- 65 
mon audio and plurality of video signals, and at pre- 
determined times, at least two selectable user options; 



a means for interacting with the integrated presentation, 
wherein the user selects a selectable user option; 

a means, connected to the interaction means, for deter- 
mining an appropriate personalized feedback response, 
wherein the feedback response may consist of video, 
audio and graphics segments and the selected feedback 
response is based on one or more user selected options; 
and 

a means, connected to the determination means, for 
presenting the appropriate personalized feedback 
response to the viewer and seamlessly switching 
between the video signals to create a visually transpar- 
ent segway between video signals, wherein the appro- 
priate personalized feedback response may occur 
immediately after user selection or at a later predeter- 
mined time in the integrated presentation. 

2. The interactive computer work station of claim 1, 
wherein the interactive computer work station further com- 
prises: 

a plurality of video sources, each source storing one of the 
plurality of video signals, wherein least one of the 
video signals contains data commands; 
a means, connected to at least one video source, for 
extracting data commands, wherein the data commands 
comprise branching codes; 
wherein the determination means comprises: 
a means for detecting the branching codes; and 
a means, connected to the detection means, for pro- 
cessing the branching codes, wherein the personal- 
ized feedback response is based on the branching 
codes and the user selected options. 

3. The interactive computer work station of claim 1 
further comprising: 

an audio card, connected to the determination means, for 
storing the plurality of different audio segments. 

4. An interactive computer work station for presenting an 
integrated multimedia interactive presentation, comprising: 

a receiver, for receiving a common video signal and a 
plurality of different video signals, wherein at least one 
of either the common video or the different video 
signals has embedded data commands, the data com- 
mands comprising branching codes and trigger points; 

a means for displaying an integrated audio, graphics and 
video presentation, the presentation comprising a com- 
mon audio and the common video signal, and at pre- 
determined times, at least two selectable user options; 

a means, connected to the receiver, for extracting data 
commands; 

a means for interacting with the integrated presentation, 
wherein the user selects a selectable user option; 

a means, connected to the interaction and extracting 
means, for determining an appropriate personalized 
feedback response, wherein the feedback response may 
consist of one of the video signals, audio and graphics 
segments and the selected feedback response is based 
on one or more user selected options and the branching 
codes, whereby the feedback responses occur at pre- 
determined trigger points, whereby the trigger point 
initiates the selection of the personalized response 
corresponding to an interactive even for presentation to 
the subscriber; and 

a means, connected to the determination means, for 
presenting the appropriate personalized feedback 
response to the viewer and seamlessly switching 
between the video signals to create a visually transpar- 
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ent segway between the video signals, wherein the 
appropriate personalized feedback response may occur 
immediately after user selection or at a later predeter- 
mined time in the integrated presentation, 

5. The interactive computer work station of claim 4 5 
further comprising: 

an audio card, connected to the determination means, for 
storing the plurality of different audio segments. 

6. An interactive computer work station for presenting an 
integrated multimedia interactive presentation, comprising: 1Q 

a means for receiving interactive programming, the inter- 
active programming comprising a plurality of video 
signals and audio signals; 

a viewer interface for receiving viewer entries; 

a microprocessor, connected to the viewer interface, for 15 
selecting one of the video and audio signals, and the 
selection of the video and audio signals is based on the 
branching codes and the received viewer entries; and 

a means, connected to the microprocessor, for presenting 
the selected video and audio signal to the viewer, 2 o 
wherein the switch to the selected video signal is 
seamless thereby creating a visually transparent seg- 
way between the video signals. 

7. A live interactive programming system, comprising: 

an interactive computer workstation for receiving live 25 
interactive programming, the live interactive program- 
ming comprising a plurality of video signals, audio 
signals, and branching codes, the workstation compris- 
ing: 

a viewer interface for receiving viewer entries; 30 
a microprocessor, connected to the viewer interface, for 
selecting one of the video and audio signals and 
directing a switch to the selected video and audio 
signals at a predetermined time, the selection of the 
video and audio signals and the predetermined time 35 
of each selection a function of the branching codes 
and the received viewer entries; 
a means, connected to the microprocessor, for switch- 
ing to the selected video signal, wherein the switch 
is seamless thereby creating a visually transparent 40 
segway between the video signals; 
a means for displaying the selected video signal; and 
a means for playing the selected audio signal. 

8. The live interactive programming system of claim 7, 
wherein the different video signals correspond to different 45 
predetermined camera angles of an event. 

9. The live interactive programming system of claim 7, 
wherein the plurality of video signals are digitally com- 
pressed. 

10. The live interactive programming system of claim 7, 50 
wherein the live programming further contains graphics 
signals and the microprocessor selects one of the graphics 
signals at a predetermined time, the selection of the graphics 
signal a function of the branching codes and the received 
viewer entries, and further comprising a means, connected to 55 
the microprocessor, for presenting the selected graphics 
signal on the display means. 

11. The live interactive programming system of claim 7, 
wherein the display means presents at least one interrogatory 

to the viewer, the content of the interrogatory involving 60 
program options, and the viewer entries corresponds to 
collected entries from the viewer via the viewer interface in 
response to the interrogatories. 

12. A live interactive digital programming system, com- 
prising: 65 

an interactive computer workstation for receiving live 
interactive programming, the live interactive program- 



ming comprising a plurality of digitally compressed 

video, audio, branching codes and graphics signals, the 

workstation comprising: 

a viewer interface for receiving viewer entries; 

a microprocessor, connected to the viewer interface, for 
selecting one of the video and audio signals and 
directing a switch to the selected video and audio 
signals at a predetermined time, the selection of the 
video and audio signals and the predetermined time 
of each selection a function of the branching codes 
and the received viewer entries, wherein the switch 
to the selected video signal is seamless so as to create 
a visually transparent segway between video signals; 

a demultiplexer, for demultiplexing the selected video 
and audio signals; 

a decompressor/decoder, connected to the demulti- 
plexer for decompressing the demultiplexed selected 
video and audio signals; 

a means for displaying the selected video signal; and 

a means for playing the selected audio signal. 

13. The live interactive digital programming system of 
claim 12, wherein the plurality of digitally compressed 
video signals corresponds to different predetermined camera 
angles of an event. 

14. The live interactive digital programming system of 
claim 12, wherein the microprocessor selects one of the 
graphics signals at a predetermined time, the selection of the 
graphics signal a function of the branching codes and the 
received viewer entries, and further comprising a means, 
connected to the microprocessor, for presenting the selected 
graphics signal on the display means. 

15. The live digital programming system of claim 12, 
wherein the display means presents at least one interrogatory 
to the viewer, the content of the interrogatory involving 
program options, and the viewer entries correspond to 
collected entries from the viewer via the viewer interface in 
response to the interrogatories. 

16. A method for presenting an integrated multimedia 
interactive presentation on a interactive computer work 
station, comprising the steps of: 

receiving an integrated audio, graphics and video 
presentation, the presentation comprising a common 
audio and a plurality of video signals, and at predeter- 
mined times, at least two selectable user options; 

interacting with the integrated presentation, wherein the 
user selects a selectable user option; 

determining an appropriate personalized feedback 
response, wherein the feedback response may consist 
of video, audio and graphics segments and the selected 
feedback response is based on one or more user 
selected options; and 

presenting the appropriate personalized feedback 
response to the viewer and seamlessly switching 
between the video signals to create a visually transpar- 
ent segway between the video signals, wherein the 
appropriate personalized feedback response may occur 
immediately after user selection or at a later predeter- 
mined time in the integrated presentation, 

17. A method for presenting an integrated multimedia 
interactive presentation on an interactive computer work 
station, comprising: 

receiving a common video signal and a plurality of 
different video signals, wherein at least one of the video 
signals has embedded data commands, the data com- 
mands comprising branching codes and trigger points; 

a means for displaying an integrated audio, graphics and 
video presentation, the presentation comprising a com- 
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mon audio and the common video signal, and at pre- 
determined times, at least two selectable user options; 
extracting data commands; 

interacting with the integrated presentation, wherein the 
user selects a selectable user option; 

determining an appropriate personalized feedback 
response, wherein the feedback response may consist 
of video, audio and graphics segments and the selected 
feedback response is based on one or more user 
selected options and the branching codes, whereby the 
feedback responses occur at predetermined trigger 
points, whereby the trigger point initiates the selection 
of the personalized response corresponding to an inter- 
active even for presentation to the subscriber; and 

presenting the appropriate personalized feedback 
response to the viewer and seamlessly switching 
between the video signals to create a transparent seg- 
way between the video signals, wherein the appropriate 
personalized feedback response may occur immedi- 
ately after user selection or at a later predetermined 
time in the integrated presentation. 

18. An interactive computer work station for presenting 
an integrated multimedia interactive presentation, compris- 
ing: 

a television, wherein the television receives an integrated 
multimedia program, the program containing a com- 
mon video signal and a plurality of different video 
signals, wherein at least one of the video signals has 
data commands, comprising: 

a means for displaying an integrated audio, graphics 
and video presentation, the presentation comprising 
a common audio and the common video signal, and 
at predetermined times, at least two selectable user 
options; and 

an interactive computer workstation, operably connected 
to the television set, comprising: 
a means for receiving the integrated multimedia pro- 
gram; 

a means, connected to the receiving means, for extract- 
ing data commands from at least one of the video 
signals, wherein the data commands comprise 
branching codes; 

a means for interacting with the integrated presentation, 
wherein the user selects a selectable user option; 
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a microprocessor, connected to the interaction and 
extracting means, for determining an appropriate 
personalized feedback response based on one or 
more user selected options and the branching codes, 
wherein if at least part of the response includes 
switching to a selected video signal, the switch to the 
selected video signal is seamless in order to create 
visually transparent segway between video signals; 
and 

a means, connected to the determination means, for 
sending commands to the television set to present on 
the display monitor the appropriate personalized 
feedback response for the viewer. 
19. A computer network for presenting an integrated 
multimedia interactive presentation, comprising: 

a video server, wherein the video server stores and pro- 
cesses interactive programs, comprising: 
means for transmitting the interactive programs; 
at least one interactive computer workstation, con- 
nected to the video server, comprising: 
a means for receiving the interactive program, the 
interactive program comprising a common audio and 
common video signal and a plurality of different 
video signals, and at predetermined times, at least 
two selectable user options; 
a means for interacting with the integrated presentation, 

wherein the user selects a selectable user option; 
a means, connected to the interaction means, for deter- 
mining an appropriate personalized feedback 
response, wherein the feedback response may consist 
of video, audio and graphics segments and the 
selected feedback response is based on one or more 
user selected options; and 
a means, connected to the determination means, for 
presenting the appropriate personalized feedback 
response to the viewer and seamlessly switching 
between the video signals to create a visually trans- 
parent segway between the video signals, wherein 
the appropriate personalized feedback response may 
occur immediately after user selection or at a later 
predetermined time in the integrated presentation. 
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